This is a cloud-based demo system where you can try TuxCare's KernelCare. In a managed environment that requires no installation, you can use the web console to try the basic usage of KernelCare's live patching.
- Internet Browser
- HTTPS access (port: 443)
- Login credentials (provided separately)
Access the API endpoint https://<server-api-endpoint> with your internet browser. A login prompt will appear. Log in to the system using the credentials provided.
The system shall be running Ubuntu 20.04.6 LTS, installed from an ISO image and has not had security updates applied. Kernel version is 5.8.0-29-generic.
$ sudo lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
$ uname -r
5.8.0-29-genericThe system is pre-configured to connect to a managed patch server. Just execute the command below for registration.
$ sudo kcarectl --register ubuntu-staging
Server Registered$ sudo kcarectl --update
Updates already downloaded
Patch level 11 applied. Effective kernel version 5.8.0-63.7120.04.1
Kernel is safe$ uname -r
5.8.0-63.7120.04.1$ sudo kcarectl --patch-info | grep kpatch-cve: | less
kpatch-cve: CVE-2020-25705
...$ sudo kcarectl --unload
KernelCare protection disabled. Your kernel might not be safe
$ uname -r
5.8.0-29-genericIn this section, we will verify that KernelCare's live patch can address vulnerabilities and pass the diagnostics of OpenScap, an open-source vulnerability scanner, without needing to reboot the system.
$ sudo kcarectl --unload
KernelCare protection disabled. Your kernel might not be safeInitially, 23 unresolved vulnerabilities are found. This is because the system has not applied security updates for Ubuntu 20.04.06. Note that the scanning may take about 30 seconds to complete.
$ oscap oval eval --report report.html com.ubuntu.$(lsb_release -cs).usn.oval.xml | grep -v oval:com.ubuntu.focal:def:100 | grep true | wc -l
23$ sudo apt-get upgrade -qq -y$ sudo apt-get install -qq -y linux-image-5.8.0-63-genericNow you should have installed all available security updates. Despite that there are still 15 remaining vulnerabilities.
$ oscap oval eval --report report.html com.ubuntu.$(lsb_release -cs).usn.oval.xml | grep -v oval:com.ubuntu.focal:def:100 | grep true | wc -l
15Access https://<server-api-endpoint>/<vm-id>/report.html
Remaining issues were reported against Kernel, because the system is still running with the old kernel 5.8.0-29-generic although a new kernel package has been installed to the filesystem. Normally, you have to reboot the system to activate the new kernel. However, in certain systems, this can cause downtime, which may be problematic.
Make sure the uname -r command reports the effective running kernel with a security updated version.
$ uname -r
5.8.0-29-generic
$ sudo kcarectl --update
Updates already downloaded
Patch level 11 applied. Effective kernel version 5.8.0-63.7120.04.1
Kernel is safe
$ uname -r
5.8.0-63.7120.04.1You will find that all vulnerabilities have been addressed without rebooting the system.
$ oscap oval eval --report report.html com.ubuntu.$(lsb_release -cs).usn.oval.xml | grep -v oval:com.ubuntu.focal:def:100 | grep true | wc -l
0Reload https://<server-api-endpoint>/<vm-id>/report.html
The column #x indicates the number of found vulnerabilities. As you can see, all vulnerabilities have been addressed and the remaining is zero.
V. Protection of Hidden Insecure Application Processes
Having installed all latest packages and applied kernel live patches, the scanner shows all green now. However, are you aware that there are still unprotected application processes existing?
There are services keep running with unpatched libraries even if you installed security patches. In order to make security patches effective, it is highly recommended to restart those services. You can use checkrestart or needs-restarting command to know which services require restarting.
$ sudo checkrestart | grep -E "^(systemctl|service)"
systemctl restart networkd-dispatcher.service
systemctl restart polkit.service
systemctl restart udisks2.service
systemctl restart clean-mount-point@.service
systemctl restart ModemManager.service
systemctl restart packagekit.service
systemctl restart packagekit-offline-update.service
service cron restart
service irqbalance restart
service atd restart
service shellinabox restart
service unattended-upgrades restart
service ssh restartCaution: checkrestart is not available in the hands-on VM
However, what if you have important services which you never want to restart? For example, Systemd and D-Bus are typical ones, since they have many dependencies, restarting them will cause significant impact on the running system. Normally, your only option would be to reboot the entire system.
TuxCare's unique LibCare can apply patches for OpenSSL and glibc in-memory to such application processes without requiring a restart.
$ sudo apt-get upgrade$ sudo kcarectl --lib-update$ sudo kcarectl --lib-info | jq | grep comm
"comm": "",
"comm": "cron",
"comm": "dbus-daemon",
"comm": "irqbalance",
...$ sudo kcarectl --lib-patch-info | jq | grep \"cve\"
"cve": "CVE-2022-4450",
...This allows the system to become more robust, and all paching works up to this point have been achieved without the need to reboot the system.
Live Patching not only serves as a countermeasure against vulnerability scanners, but also actually prevents exploits, as it can be demonstrated by running the POC code of the CVE-2022-0847 as known as DirtyPipe. Note that this demonstration is not available in the managed system but it can be requested separately.
- CVE-2022-0847 DirtyPipe Exploit (https://github.com/febinrev/dirtypipez-exploit)
Credit: Max Kellermann and Febin Mon Saji
$ whoami
guest
$ id
uid=1001(guest) gid=1001(guest) groups=1001(guest)
$ su
Password:
su: Authentication failure$ sudo kcarectl --unload
KernelCare protection disabled. Your kernel might not be safe
$ uname -r
5.8.0-29-genericThe attack succeeds and root privileges are taken.
$ dirtypipez
[+] hijacking suid binary..
[+] dropping suid shell..
[+] restoring suid binary..
[+] popping root shell.. (dont forget to clean up /tmp/sh ;))
# # whoami
root
# id
uid=0(root) gid=0(root) groups=0(root),1001(guest)# rm /tmp/sh
# exit
$$ sudo kcarectl --update
Downloading updates
Patch level 11 applied. Effective kernel version 5.8.0-63.7120.04.1
Kernel is safe
$ uname -r
5.8.0-63.7120.04.1$ sudo kcarectl --patch-info | grep " CVE-2022-0847"
kpatch-cve: CVE-2022-0847This time kernel shall block the attack to escalate privileges.
$ dirtypipez
[+] hijacking suid binary..
[+] dropping suid shell..
usage: sudo -h | -K | -k | -V
usage: sudo -v [-AknS] [-g group] [-h host] [-p prompt] [-u user]
usage: sudo -l [-AknS] [-g group] [-h host] [-p prompt] [-U user] [-u user] [command]
usage: sudo [-AbEHknPS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p prompt] [-T timeout] [-u user] [VAR=value] [-i|-s] [<command>]
usage: sudo -e [-AknS] [-r role] [-t type] [-C num] [-g group] [-h host] [-p prompt] [-T timeout] [-u user] file ...
[+] restoring suid binary..
[+] popping root shell.. (dont forget to clean up /tmp/sh ;))
sh: 1: /tmp/sh: not found
$Demonstrate Libcare can protect running application processs from known CVEs by applying patches in memory. Use CVE-2023-4813 as an example.
- https://nvd.nist.gov/vuln/detail/CVE-2023-4813 (getaddrinfo)
Test Code: https://github.com/tnishiox/cve-2023-4813
Reboot the system if you have installed the latest 'libc6' package to reproduce the vulnerability of CVE-2023-4813.
$ sudo reboot -f$ sudo kcarectl --register ubuntu-staging
Server RegisteredThe test code 'cve-2023-4813' will cause a segfault if the system is vulnerable.
$ cve-2023-4813 example.org
Segmentation fault (core dumped)Any applications calling 'getaddrinfo' can cause a crash if you have a certain configuration in '/etc/nsswitch.conf'. For example even 'ping' command can crash.
$ cat /etc/nsswitch.conf | grep host
hosts: dns [SUCCESS=continue] filesThis time add delay at startup by giving the 2nd argmnent, and also run the program in background.
$ cve-2023-4813 example.org 30 &
[1] 6023
$ Wait for 30 seconds.
Run 'sudo kcarectl --lib-update' in the meantime.By doing that 'glibc' loaded by 'cve-2023-4813' process can be fixed in memory.
$ sudo kcarectl --lib-update
The patches have been successfully applied to 33 newly discovered processes. The overall amount of applied patches is 49.
Object `libc-2.31.so` is patched for 33 processes.
Object `libcrypto.so.1.1` is patched for 16 processes.
Userspace patches are applied.You should be able to confirm the application avoids crash and exit without problem this time.
$ getaddrinfo: Name or service not known
[1]+ Exit 2 cve-2023-4813 example.org 30Note that 'Name or service not known' error is expected result in this case.
Rebooting lets the system discard all changes and reset to its initial state.
$ sudo reboot -f-
Available Shell Commands
Users are restricted to a limited bash shell and can only execute permitted commands.
$ ls bin | grep -v .sh apt apt-get cat checkrestart cve-2023-4813 grep id jq kcarectl kcare-scanner-interface less ls lsb_release oscap realpath reboot sudo uname w3m wc whoami -
Users can access only the home directory.
-
Sudo can be executed with only permitted commands.
-
Only one user can login at a time.
-
None of the file transfer protocols (scp, sftp, rsync) is permitted.
-
System will automatically reset if a user remains logged in for an extended period.
-
Target system regularly reboots to reset to its initial state regardless of the login state.